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ABSTRACT 

Operational  Analysis  (OA)  supports  a  wide  range  of  military  decisions,  from  tactical  level  through  to 
campaign  implementation.  As  all  military  operations  are  human  activities,  there  is  a  requirement  to 
deliver  appropriate  representation  of  human  issues  in  OA  advice  to  UK  Ministry  of  Defence  (MOD) 
decision-makers.  It  has  long  been  acknowledged  that  in  a  number  of  OA  models,  containing  humans  with 
group  behaviours,  there  is  insufficient  consideration  of  key  human  factors  (HF)/human  issues.  A  reason 
given  by  the  OA  community  is  that  the  Human  Sciences  (HS)  community  do  not  have  the  required  data  in  a 
readily  useable  format  for  their  OA  models.  Correspondingly  the  HS  community  have  argued  that  the 
analysts  do  not  adequately  define  their  requirements.  Defence  Science  &  Technology  Laboratory  (Dstl) 
was  tasked  to  identify  a  common  set  ofHF  to  be  considered  for  inclusion  within  OA  models,  and  to  review 
and  capture  the  key  interfaces  and  issues  between  the  OA  and  HS  communities.  This  latter  task  aimed  to 
continue  to  improve  the  working  relationship  between  the  two  communities  via  a  process  promoting  a 
shared  understanding,  and  flow  of  information  and  data  between  the  communities.  The  study  determined 
that  there  was  no  common  set  of  human  factors  for  inclusion  within  OA  models.  The  key  set  of  HF  was 
dependent  upon  the  context,  the  question  to  be  addressed,  and  the  model’s  level  of  abstraction.  A  draft 
Good  Practice  Guide  was  developed  to  capture  an  auditable  process  by  which  the  OA  and  HS 
communities  could  exploit  each  others  areas  of  expertise  in  order  to  inform  their  respective  research 
programmes.  This  was  validated  and  refined  using  existing  OA  models. 


1  INTRODUCTION 

Operational  Analysis  (OA)  supports  a  wide  range  of  military  decisions,  from  tactical  level  through  to 
campaign  implementation.  As  all  military  operations  are  essentially  driven  by  human  activities,  there  is  a 
requirement  to  deliver  appropriate  representation  of  these  human  issues  in  OA  advice  to  Ministry  Of 
Defence  (MOD)  decision-makers. 

The  following  research  study  was  designed  to  address  the  UK  MOD  requirement  to  develop  improved  OA 
model  outputs  by  improving  the  representation  of  Human  Factors  (HF1).  The  initial  stages  of  research 
were  undertaken  over  the  period  from  January  2006  until  March  2009.  The  detail  of  the  study  can  be 
found  at  Ref  [1], 

This  paper  provides  an  overview  of  the  study  activities  and  summarises  the  key  outcomes  and  exploitation. 


1  Human  Factors  (HF)  refers  to  the  academic/theoretical/practical  understanding  of  the  human  in  the  system,  and  its  application. 


RTO-MP-HFM-202 


13-1 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

OCT  2010 


2.  REPORT  TYPE 


3.  DATES  COVERED 


4.  TITLE  AND  SUBTITLE 


Improving  the  Representation  of  Human  Factors  in  Operational  Analysis 


6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Defence  Science  &  Technology  Laboratory  (Dstl)  Policy  &  Capability 
Studies  Dept  Level  C-K  West  Court  Portsdown  West  Portsdown  Hill 
Road  Fareham  PO!7  6AD  UK 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 


5c.  PROGRAM  ELEMENT  NUMBER 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 


13.  SUPPLEMENTARY  NOTES 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 


See  also  ADA564696.  Human  Modelling  for  Military  Application  (Applications  militaires  de  la 
modelisation  humaine).  RTO-MP-HFM-202 


14.  ABSTRACT 
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group  behaviours,  there  is  insufficient  consideration  of  key  human  factors  (HF)/human  issues.  A  reason 
given  by  the  OA  community  is  that  the  Human  Sciences  (HS)  community  do  not  have  the  required  data  in 
a  readily  useable  format  for  their  OA  models.  Correspondingly  the  HS  community  have  argued  that  the 
analysts  do  not  adequately  define  their  requirements.  Defence  Science  &  Technology  Laboratory  (Dstl)  was 
tasked  to  identify  a  common  set  of  HF  to  be  considered  for  inclusion  within  OA  models,  and  to  review  and 
capture  the  key  interfaces  and  issues  between  the  OA  and  HS  communities.  This  latter  task  aimed  to 
continue  to  improve  the  working  relationship  between  the  two  communities  via  a  process  promoting  a 
shared  understanding,  and  flow  of  information  and  data  between  the  communities.  The  study  determined 
that  there  was  no  common  set  of  human  factors  for  inclusion  within  OA  models.  The  key  set  of  HF  was 
dependent  upon  the  context,  the  question  to  be  addressed,  and  the  models  level  of  abstraction.  A  draft 
Good  Practice  Guide  was  developed  to  capture  an  auditable  process  by  which  the  OA  and  HS  communities 
could  exploit  each  others  areas  of  expertise  in  order  to  inform  their  respective  research  programmes.  This 
was  validated  and  refined  using  existing  OA  models. 
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2  MILITARY  NEED 

Where  OA  models  contain  humans  and  systems  with  group  behaviours,  then  they  should  also  include  key 
human  factors  and  assumptions  about  these  behaviours.  This  should  then  enable  the  model  to  represent  the 
differing  behaviours  which  could  influence  operational  effectiveness  and  the  analyst  could  use  the  data  to 
represent  these  factors  in  the  OA  models.  Ideally  any  empirical  data  would  have  been  reviewed  and  any 
strengths  and  weaknesses  recorded.  However,  human  factors  are  not  simple  to  represent  in  the  OA  models 
as  their  very  nature  means  that  there  will  be  differing  degrees  of  inter -dependency  between  them.  This 
then  increases  the  complexity  when  attempting  to  produce  any  quantitative  data  for  use  in  the  OA  models. 

There  is  a  recognised  problem  that  OA  models  do  not  always  adequately  take  account  of  operationally 
significant  Human  Sciences  (HS2)  issues.  There  have  been  a  number  of  reasons  for  this.  The  analysts 
argue  that  the  HS  community  do  not  have  the  required  level  of  definitive  data  on  the  behaviour  of  humans 
in  complex  systems  and  networks.  The  HS  community  argues  that  the  analysts  do  not  define  their 
requirement  in  enough  detail  for  it  to  be  addressed. 

The  following  example  of  an  email  request,  from  an  analyst  to  the  HS  community  for  support  to  an  OA 
model,  neatly  captures  the  military  need: 


“Hi 


I  was  wondering  whether  you  or  your  various  colleagues  in  HS  could  give  us  a  bit  of  a  help  by  providing 
some  information  for  our  model.  I  hope  I've  come  to  the  right  person.  We're  currently  in  the  process  of 
verifying  and  validating  a  model  and  we're  trying  to  include  some  soft  effects  and  human  factors  in  the 
mix.  We're  particularly  looking  for  data/insights  on  the  effect  of  fatigue,  familiarisation  with  the  terrain, 
experience  and  leadership  on  battlegroup  sized  operations  (particularly  tempo  and  mission  effectiveness). 

If  you've  got  any  data  to  hand  along  those  lines,  it  would  be  great, 

Many  thanks, 

(OA  Modeller)  ” 

This  email  highlighted  the  following  three  main  issues,  all  of  which  have  been  brought  into  sharp  focus 
over  the  course  of  the  study: 

•  First,  that  a  model  has  been  built  in  answer  to  a  customer  question  and  is  at  the  ‘validation’  stage 
yet  only  post  this  stage  is  HS  input  sought. 

•  Second  that  a  non-HS  specialist  has  already  decided  which  HF  are  relevant  to  the  model  (though 
this  would  generally  have  been  informed  through  Historical  Analysis). 

•  Third,  that  there  is  an  explicit  assumption  that  the  HS  community  can  respond  meaningfully,  when 
no  context  is  given  to  the  problem  space  or  definition  provided  of  the  data  requirement  and  how  it 
will  be  incorporated. 

The  HS  community  simply  cannot  answer  this  request  for  help  meaningfully  given  these  constraints  and 
would  need  far  more  information  to  bound  the  question. 

Conversely  the  OA  community  are  continually  frustrated  by  the  perceived  inability  of  the  HS  community 
to  provide  a  straight  forward  response  to  their  queries.  HS  responses  tend: 


2 

Human  Science/Scientist  (HS)  refers  to  the  scientific  disciplines  and  community  of  experts  practicing  in  and  with  expert 
knowledge  of  HF. 


13-2 


RTO-MP-HFM-202 


Improving  the  Representation  of  Human  Factors  in  Operational  Analysis 


•  To  be  overly  complex 

•  To  lack  supporting  information/data  in  a  format  is  easily  used  to  inform/support  an  OA  query 

•  To  lack  (in  the  absence  of  any  available  theory/previous  research)  provision  of  an  informed 
opinion  that  could  then  be  followed-up  with  more  grounded  data 

In  addressing  the  study  requirement,  a  review  of  the  balance  of  issues  on  either  side  was  required  in  order 
to  continue  to  improve  the  working  relationship  between  the  communities. 

There  is  a  clear  need  to  forge  stronger  understanding  between  the  OA  and  HS  communities,  in  order  to 
identify  and  understand  the  human  and  system  group  behaviours  involved,  the  Human  Sciences  factors 
which  influence  them,  and  then  develop  a  plan  to  improve  their  representation  in  OA  models.  The 
improved  models  will  in  turn  further  de-risk  responses  to  future  UK  MOD  customer  tasking. 


3  STUDY  PURPOSE  AND  APPROACH 

The  purpose  of  the  study  was  to: 

•  Develop  an  effective  collaborative  working  relationship  between  the  OA  and  HS  communities. 

•  Identify  and  agree  a  common,  small  set  of  key  human  aspects  and  factors,  which  influence 
networks  or  systems  that  have  group  behaviour,  for  inclusion  in  OA  models 

•  Provide  a  basis  upon  which  to  focus  the  future  HS  research  programme. 

This  study  was  a  collaborative  activity  between  the  OA  and  the  HS  community  and  was  based  on  an 
iterative  principle,  whereby  results  from  each  stage  were  validated  and  of  value  in  their  own  right,  but  also 
informed  the  next  stages. 

The  study  approach  was  broken  down  into  the  following  broad  tasks: 

•  Task  1:  Identify  the  key  representatives  from  each  community  to  contribute  to  the  collaboration, 
define  the  problem  space  from  the  OA  domain  perspective  and  identify  the  key  group  behaviours 
and  corresponding  HF  involved 

•  Task  2:  Prioritise  the  HF  identified  in  Task  1,  by  the  greatest  level  of  impact  to  Operational 
Effectiveness,  and  then  identify  a  common,  small  subset  of  variables  for  use  in  the  OA  models. 
(This  was  amended  to  create  a  framework  with  key  HF  themes  through  which  the  HF  relevant  to 
the  customer  question  could  be  examined) 

•  Task  3:  Using  the  framework  developed  in  Task  2,  identify  and  agree  relevant  HF  influences  in  a 
use  case,  i.e.  peace  support  OA  model  (DIAMOND3).  Then  determine  the  availability  of  data  to 
represent  the  HF  identified  in  the  model.  This  task  would  also  capture  the  key  interfaces  and 
issues  between  the  OA  and  HS  communities. 

The  outcomes  for  the  above  tasks  together  with  lessons  learned  were  then  distilled  into  activities  of  good 
practice  and  captured  in  a  draft  good  practice  guide  (GPG)  as  part  of  the  research  exploitation  outcomes. 
This  draft  GPG  was  devised  for  application  across  the  OA  community  by  the  analysts  in  order  to  facilitate 
the  consideration/inclusion  of  HF  in  OA  models. 

The  original  basis  of  the  study  was  extended  in  order  to  further  test  and  validate  this  process  as  follows: 

•  Task  4:  Using  lessons  learned  from  Task  3,  repeat  the  theme  based  workshop  to  identify  and  agree 
those  influences  important  to  DIAMOND,  amend  the  process  and  update  the  GPG  as  required. 


3 

'  DIAMOND  -  Diplomatic  And  Military  Operations  in  a  Non-warfighting  Domain 
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•  Task  5:  Repeat  the  theme  based  workshop  to  identify  and  agree  HF  for  inclusion  for  a  war  fighting 
OA  model  (COMAND4).  Again,  from  the  outcomes,  refine  the  GPG  with  regard  to  application  to 
other  (war  fighting)  models 

As  the  study  progressed  and  findings  emerged,  the  boundaries  between  these  tasks  became  somewhat 
blurred,  as  practical  considerations  determined  the  most  pragmatic  ways  forward. 


4  STUDY  METHOD  AND  ASSUMPTIONS 

The  majority  of  the  tasks  used  workshops  to  achieve  their  results.  Where  wider  consultation  across  Dstl 
was  required  a  multi-disciplinary  collaborative  group  provided  guidance.  Wider  awareness  and 
benchmarking  against  other  HF  modelling  initiatives  was  conducted  for  cross-checking  and  validation. 
The  inclusion  of  military  personnel  at  the  workshops  was  essential  and  invaluable  in  the  generation  of 
ideas,  and  the  provision  of  advice,  to  ensure  consistency  with  Operations. 

It  was  assumed  that  for  the  study  to  be  completed  successfully,  the  participation  of  representatives  from 
both  the  OA  and  HS  communities  was  essential.  In  addition  this  task  was  to  draw  upon  extant 
information/data  regarding  HF;  therefore,  the  development  of  new  HF  research  was  outside  the  remit  of 
this  study. 


5  STUDY  OUTCOMES 

The  following  section  outlines  the  activities  of  each  of  the  above  tasks  together  with  corresponding 
outcomes. 

5.1  Task  1:  Identify  a  Collaborative  Group,  Define  the  OA  Problem  Space,  Identify  the 
Key  HF  Influences  Involved 

This  task  comprised  the  following  three  main  elements: 

•  Identifying  members  for  a  ‘Collaborative  Group’  to  contribute  to  the  research 

•  Defining  the  OA  problem  space 

•  Identifying  the  group  behaviours  and  corresponding  influences  within  the  OA  problem  space. 
Facilitated  workshops  were  used  to  illicit  and  capture  the  required  information  for  use  in 
subsequent  workshops. 

5.1.1  Task  1.1  Identification  of  Key  Contributors  to  the  Study 

The  first  workshop  brought  together  a  wide  variety  of  possible  interested  parties  within  Dstl.  From  this,  a 
group  of  both  OA  and  HS  community  Subject  Matter  Experts  (SMEs)  were  identified  who  would  be 
willing  to  become  part  of  a  ‘Collaborative  Group’.  This  group’s  remit  was  to  provide  oversight  and 
guidance  on  the  study  and  where  appropriate  ensure  coherence  with  other  related  developments  or  studies 
within  the  two  communities  and  across  Dstl. 

This  group  initially  comprised  two  OA  specialists,  two  HS  specialists,  and  two  with  experience  across 
both  areas.  Although  the  profile  of  the  Group  changed  over  the  course  of  the  study,  it  was  an  effective  and 
powerful  source  of  advice  and  provided  assurance  of  scientific  rigour  throughout  the  review  process.  In 
addition  to  this  Collaborative  Group,  a  review  was  also  undertaken  by  both  military  personnel  and  a 
member  of  Dstl’ s  Historical  Analysis  team,  to  ensure  coherence  and  validity  of  the  output. 


4  COMAND  -  C3  Orientated  Model  of  the  Air  and  Naval  Domain 
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This  first  workshop  also  sought  information  from  participants  on  similar  work  that  has  been,  or  was 
currently  being,  undertaken  in  this  area.  This  was  followed  up  throughout  the  duration  of  the  study  via  a 
Study  Coherency  Group  [Ref  2]. 

5.1.2  Task  1.2  Definition  of  the  OA  Domain  Problem  Space 

Using  members  from  the  Collaborative  Group,  a  second  workshop  was  arranged  to  develop  a  mutually 
agreed  understanding  of  the  real  world  problem  space  addressed  by  the  OA  domain,  and  identify  the 
systems  behaviours  and  HF  involved.  The  aim  of  the  second  workshop  was  quite  ambitious  not  only  in 
terms  of  the  desired  output;  but  also,  in  the  novelty  of  the  approach;  as  this  was  the  first  workshop 
bringing  representatives  from  the  two  communities  together  to  achieve  a  relatively  concrete  output  from 
unfamiliar  perspectives  and  constructs.  There  were  likely  to  be  fundamental  differences  of  understanding 
and  language  between  those  from  each  community.  It  was  therefore  important  to  establish  a  framework  of 
shared  understanding  upon  which  the  workshop  participants  could  build. 

To  this  end,  the  core  team  sought  advice  from  the  OA  community  on  how  best  to  represent  the  MOD 
problem  space  addressed  by  the  OA.  For  this  study  the  MOD  problem  space  was  bounded  by  timeliness 
and  operational  type.  Timeliness  was  viewed  in  terms  of  Force  Readiness  and  operational  type  provided 
an  idea  of  the  severity  and  degree  of  involvement  in  military  tasks.  Both  concepts  are  familiar  and 
employed  by  the  military  Ref  [2] .  This  framework  served  to  assist  the  HS  community  in  understanding  the 
OA  problem  space  and  was  agreed  to  encompass  the  types  of  OA  models  that  currently  exist  or  are  likely 
to  be  developed. 

Having  agreed  a  framework  there  was  now  a  need  to  bridge  the  gap  between  this  overarching  OA  domain 
problem  space  and  a  specific  context/problem  space  that  an  OA  model  would  normally  be  seeking  to 
address.  It  was  therefore  agreed  that  a  set  of  Phenomena5  should  be  generated  to  provide  this  context.  In 
addition,  a  series  of  Tenses’  representing  differing  HS  perspectives,  should  be  used  to  identify  HF  which 
have  an  influence  within  these  Phenomena  and  for  this  the  Physical,  Information,  Cognitive, 
Organisational  and  Social  (PICOS)  Domains6  were  employed. 

5.1.3  Task  1.3  Identification  of  Group  Behaviours  and  Corresponding  Influences 

This  process  saw  the  generation  of  many  ideas,  some  of  which  were  subsets  or  underlying  influences  of 
others.  The  military  input  to  this  stage  was  particularly  valuable  as  it  allowed  an  injection  of  ‘real  world’ 
operational  experience.  Participants  then  agreed  which  of  these  ideas  could  be  expressed  as  ‘Phenomena’ 
and  taken  forward  into  the  next  stage.  The  following  six  key  ‘Phenomena’  were  identified: 

•  Achieve/lose  control  of  pace  of  operations 

•  Change  in  support 

•  Gain/loss  of  force  cohesion 

•  Failure  of  force  elements  to  participate  in  the  action 

•  Fratricide  and  collateral  damage 

•  Surprise. 

From  the  above  six  Phenomena  identified  five  were  chosen  by  the  group  to  take  forward  to  the  next  stage. 
Surprise  was  omitted  as  it  was  agreed  that  this  context  was  already  covered  by  the  five  Phenomena  put 


5  Phenomena;  ‘behaviour  or  state  changes  attributable  to  the  presence  of  humans  in  the  system,  which  can  readily  be  related  to  a 
value  construct  such  as  effectiveness’.  E.g  Loss  of  Force  Cohesion. 

6  PICOS  Domains  were  developed  from  an  idea  in  the  US  Network  Centric  Conceptual  Model.  These  domains  are  defined  at 
[Ref  2;  Annex  A], 
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forward.  Full  definitions  for  the  above  Phenomena  (as  generated  by  the  workshop  participants)  are 
provided  a  [Ref  2;  Annex  B],  Using  the  Phenomena  identified  together  with  the  PICOS  domains  over  a 
hundred  HF  influences  were  generated. 

During  the  workshop,  assumptions  and  issues  were  recorded  however  the  most  important  observation  was 
the  issue  of  definitions.  Whilst  definitions  were  provided  for  PICOS,  Phenomena  etc.  the  study  team  found 
that  these  and  other  terms  had  very  different  meanings  across  the  two  communities.  Within  the  project  too, 
it  became  important  to  understand  clearly  what  was  meant  by  a  ‘human  factor’,  FIS,  and  an  ‘Influence’  and 
other  terms  defined  here. 

The  outcome  of  this  workshop  was  recorded  and  participants  were  given  the  opportunity  to  comment  on 
and  amend  the  output.  The  most  important  reason  for  this  was  that  the  facilitator  where  possible  included 
definitions  or  descriptors  (including  illustrative  examples  where  necessary)  of  these  HF  Influences  based 
on  the  discussions  observed  in  the  workshop.  Participants  were  asked  to  verify  these  and  add  descriptors 
where  these  had  not  been  adequately  captured.  These  descriptors  proved  invaluable  in  the  later  stages  of 
the  research. 

In  terms  of  taking  these  findings  forward,  it  had  become  evident  that  the  HF  influences  identified  were 
very  dependent  on  both  context  (the  problem  space  and  the  question  the  model  was  seeking  to  answer)  and 
level  of  abstraction.  As  a  result,  it  became  clear  that  it  was  not  going  to  be  possible,  as  originally  hoped,  to 
identify  a  common  subset  of  HF  to  be  considered  when  developing  any  OA  model. 

5.2  TASK  2  Prioritise  the  HF  and  Identify  a  Small  Common  Set  of  HF/Human  Issues  for 
OA  Models 

In  consultation  with  the  Collaborative  Group,  Task  2  comprised  three  main  activities;  consultation 
exercise,  a  theming  exercise  and  trial  workshop.  A  summary  of  these  and  their  outcomes  are  described 
below. 

5.2.1  Task  2.1  Consultation  Exercise 

The  agreed  table  of  HF  Influences:  post  Workshop  [Ref  2;  Annex  C]  captured  in  Task  1.3  was 
promulgated  widely  across  Dstl  for  comment.  Overall,  although  some  additions  were  made  to  the 
influences  identified,  very  few  were  deleted  or  moved.  The  table  of  influences  including  those 
contributions  from  the  consultation  exercise  is  detailed  in  [Ref  3;  Annex  B].  It  became  apparent  that  not 
only  did  the  additions  seem  to  show  greater  commonality  across  the  Phenomena,  but  also,  that  the 
descriptors  given  by  contributors  were  key  to  informing  individual’s  assessment.  For  example,  one 
contributor  identified  leadership  as  a  factor.  From  the  description  provided  it  became  apparent  that  the 
issue  at  hand  was  the  individual  commander’s  willingness  to  make  a  decision  based  on  limited 
information  and  his/her  attitude  to  risk  taking.  This  fell  far  more  into  the  area  of  decision  making  and 
situational  awareness. 

Although  contributors  were  asked  to  assign  a  priority  to  the  HF  generated  within  each  domain  depending 
on  perceived  impact  on  operational  effectiveness,  this  was  only  achieved  with  moderate  success;  again 
most  felt  unable  to  assign  a  generic  importance  to  any  one  factor,  without  greater  awareness  of  the  context. 
The  only  criticism  that  emerged  was  the  land  centric  focus  of  many  of  the  examples.  This  was  in  part  due 
to  the  context  offered  by  the  original  example  Phenomena  but  the  point  was  noted  as  the  team  moved  into 
the  next  stage  of  the  study. 

5.2.2  Task  2.2  Theming  Exercise 

On  closer  inspection  by  the  study  team,  many  of  the  influences  identified  appeared  to  start  falling  into 
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natural  groupings,  irrespective  of  the  OA/Phenomena  frameworks  in  which  they  had  been  generated. 
Rather  than  identify  a  subset  of  common  key  HF,  it  became  clear  it  would  be  more  appropriate  to  sort  the 
HF  influences  into  emerging  themes.  Using  an  iterative  process,  13  HF  themes  were  identified  (of  which 
there  were  two  types;  Intrinsic7  and  Extrinsic8).  On  completion  of  the  ‘theming’  process,  titles  and  draft 
definitions  for  each  theme  were  generated,  which  were  deemed  to  encapsulate  the  essence  of  the 
underpinning  influences.  A  table  of  the  themes  and  their  definitions  was  produced.  This  table  was 
subsequently  reviewed  by  the  Study  Coherency  Group  and  military  personnel,  before  being  tested  using 
existing  OA  models.  The  table  of  themes  was  further  reviewed  following  each  user  case  study  (Tasks  3,  4 
and  5).  Before  the  final  version 

5.2.3  Task  2.3  Trial  Workshop  -  Customer  Question 

As  an  additional  task,  during  the  consultation  exercise,  a  small  trial  workshop  was  undertaken.  This  was  to 
begin  to  assess  the  potential  in  terms  of  collaboration  between  the  OA  and  HS  communities  at  the  earliest 
stage  in  the  process  of  addressing  a  customer  question.  In  this  case  a  hypothetical  customer  question  was 
proposed  by  the  OA  community,  two  of  whom,  together  with  four  from  the  HS  community,  took  part  in  a 
workshop  to  assess  how  a  particular  HF  (fatigue)  would  impact  on  Operational  Effectiveness  (OE)  in  a 
particular  task.  The  objective  here  was  to  determine  if  the  HS  participants  could  identify  and  prioritise  the 
most  important  HF  elements  of  fatigue  and  advise  the  OA  community  on  the  availability  and  maturity  of 
data  that  could  be  provided  for  input  into  such  a  question. 

From  this  workshop,  several  key  lessons  were  identified.  These  included  the  need  of  the  HS  community  to 
understand  fundamental  and  underpinning  information  such  as  the  purpose  of  the  question,  who  was  likely 
to  use  the  answer  and  to  what  puipose.  Also,  there  was  a  need  for  the  OA  community  or  customer  to 
bound  the  problem  in  terms  of  environment  (altitude  and  climate),  time  of  troops  in  theatre 
(acclimatisation  etc),  force  fitness  (injuries)  etc.  Where  the  answer  was  not  known,  an  assumption  had  to 
be  made  (and  recorded).  Through  this,  the  OA  community  were  able  to  appreciate  that  there  was  such  a 
wealth  of  data  and  information  available,  across  such  a  wide  variety  of  disciplines  within  the  overarching 
umbrella  of  HS,  that  the  problem  had  to  be  narrowed  down  to  allow  meaningful  engagement  with  the  HS 
community.  On  the  part  of  the  HS  community,  they  were  also  able  to  ‘park’  various  issues  or  unknowns, 
in  order  to  provide  input.  Again,  a  key  element  of  this  process  was  language,  and  the  mutual 
understanding  of  terms. 

Throughout  the  conduct  of  the  study,  it  was  becoming  increasingly  clear,  that  the  original  Task  Three 
which  sought  to  review  the  current  maturity  and  availability  of  data  for  the  OA  models,  and  the  level  of 
representation  of  the  identified  HS  factors  in  existing  OA  models,  was  both  impractical  and  unlikely  to  be 
cost  effective.  The  anticipated  common  subset  of  HF  that  should  be  incorporated  into  all  OA  models 
simply  did  not  exist.  Instead  the  study  had  derived  key  themes  or  umbrellas  under  which,  depending  on 
the  context  of  the  OA  problem  space,  particular  HF  would  come  into  play,  at  varying  levels  of  importance 
and  with  many  interdependencies.  The  HF  would  also  have  an  impact  at  differing  levels  within  model 
structures.  In  agreement  with  the  customer,  Task  3  was  revised  to  take  account  of  these  findings. 

5.3  Task  3  Test  Theme  Framework  Developed  in  Task  2,  Using  an  Example  OA  Model 

This  revised  task  gave  the  study  team  the  opportunity  to  validate  both  the  themes  and  process  of 
engagement  of  the  two  communities  and  to  build  on  the  lessons  identified  to  produce  the  GPG.  For  this 
task  a  peace  support  campaign  OA  model  (DIAMOND)  was  used.  This  task  comprised  three  key  elements 
and  their  description  and  outcomes,  are  outlined  below. 


7  Intrinsic  Themes  -  Groups  of  Influences,  which  pertain  to  human  skills,  abilities  and  behaviours  such  as  decision  making. 
These  themes  are  measurable  with  existing  data  and  assumptions. 

S  Extrinsic  Themes  -  Groups  of  Influences,  which  provide  the  means  or  capability  to  perform  these  human  skills,  abilities  and 
behaviours  such  as  Training.  These  themes  are  not  easily  measurable/quantifiable. 
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5.3.1  Task  3.1  Validation  of  Themes 

This  was  undertaken  both  by  external  (to  the  project)  benchmarking  against  other  HF  related  model 
development  and  a  cross  checking  of  theme  definitions  against  military  meanings.  The  benchmarking 
exercise  sought  to  ensure  some  degree  of  commonality  in  both  outcome  and  lessons  identified.  The  cross 
checking  exercise  aimed  to  ensure  that  theme  names  and  definitions  were  in  tune  with  common 
understanding  of  those  terms  particularly  for  the  Armed  Forces.  Had  the  title  and  definition  of  a  theme 
been  in  conflict  with  an  existing  understanding,  alternative  titles  would  have  been  found,  an  example 
being  Doctrine. 

5.3.2  Task  3.2  Testing  the  Process 

A  workshop  was  held  with  four  HS  participants  (with  disciplines  relevant  to  the  problem  space),  two 
analysts  and  a  Military  Advisor  (MA).  None  of  the  participants,  except  the  MA,  had  taken  part  in  any  of 
the  previous  workshops.  Building  on  the  lessons  identified  in  the  earlier  workshop,  the  HS  representatives 
were  provided  with  a  detailed  briefing  on  the  OA  model  before  the  workshop.  The  workshop  tasked 
participants  to  consider  the  HF  relevant  to  an  existing  OA  model  using  the  themes  as  a  framework.  It  was 
the  intention  that  HF  identified  would  then  be  clarified,  a  comparison  made  against  those  already  included 
in  the  model,  and  a  priority  agreed  on  those  HF  that  should  be  taken  forward  for  inclusion  or  amendment 
in  the  model.  Unfortunately,  the  latter  elements  of  this  activity  could  not  be  undertaken  due  to  time 
constraints,  which  in  itself  provided  further  valuable  lessons  to  be  recorded.  However,  a  number  of  HF 
were  identified  which  gave  some  initial  direction  to  the  OA  model  custodians  on  the  validity  of  current  HF 
within  the  model,  and  any  that  might  need  to  be  considered  in  the  future  . 

The  overall  outcome  was  that  the  process  in  principle  was  a  success  and  the  themes  offered  structure  and 
inspiration  in  generating  ideas.  The  key  lessons  learned  however,  included  the  need  to  ensure  precision 
from  the  HS  participants  (e.g.  the  word  ‘morale’  as  a  factor  was  insufficient,  more  detailed  information 
was  required),  to  ensure  the  OA  (and  thus  non  HS  expert)  contributions  were  identified  as  such,  and  to 
maintain  a  fast  pace  throughout  the  day  to  ensure  the  objectives  could  be  achieved.  The  one  criticism 
made  of  the  theme  definitions  was  their  apparent  focus  on  ‘Blue’  own  forces,  rather  than  ‘Red’  opposing 
forces  or  any  other  neutral  (Green)  or  unknown  (Yellow)  actors.  This  again  was  taken  on  board  for  future 
reference,  though  again,  will  be  dependent  on  the  context  of  the  customer  question.  Following  these 
exercises  the  themes  and  their  definitions  were  amended  slightly  to  reflect  feedback.  Some  themes  may  be 
irrelevant  to  a  given  problem  space,  model  or  question,  however  that  they  are  considered  offers  confidence 
that  critical  HF  are  not  omitted.  The  final  list  of  themes  and  their  definitions  are  shown  in  the  Table  1 
below. 
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Table  1 :  Final  Themes  and  Definitions. 

Themes 

Definition 

Decision  Making 

The  willingness  and  ability  to  decide  on  an  appropriate  course  of  action  (e.g.  minimising  risk 
to  own  participants)  based  on  changeable  situational  awareness. 

Doctrine 

Military  doctrine  articulates  the  fundamental  principles  that  guide  how  our  military  forces 
conduct  their  actions.  It  is  authoritative  but  requires  judgement  in  its  application.  The 
principal  purpose  of  doctrine  is  to  provide  a  framework  of  guidance  for  the  conduct  of 
military  operations,  but  it  also  serves  to  underpin  our  training  and  preparation.  It  is  about  how 
operations  should  be  directed,  mounted,  commanded,  conducted,  sustained  and  recovered. 

Force  Fitness 

The  ability  of  the  force  element9  to  complete  the  operational  task111  given  the  demands  of 
fatigue,  illness  or  injury. 

Flost  Population  Belief 

Structure11 

Thresholds  of  acceptability  to  host  population’s  culture  of  own  forces  activities. 

Logistical  Support 

The  availability  of  the  right  resources,  including  functioning  equipment  and  supplies,  at  the 
right  place  at  the  right  time  for  the  required  tasks,  and  the  capability  to  sustain  this  support. 

Motivation 

The  desire* 1-  of  a  force  element  to  strive  towards  completing  the  required  task/common  goal. 

Perceptions  of  Host  Population 

The  host  population’s  perception  of  how  key  aspects  of  their  life  have  improved  (or 
worsened)  subsequent  to  non  host  forces  activities. 

Personnel  Strategy 

Overarching  strategy  outlining  MoD  personnel  policies  and  processes  including  level  of 
support  given  to  families  and  service  personnel. 

Quality  of  Information 

The  usefulness  of  the  information  available  (quality,  credibility  and  timeliness)  to  inform 
decision  making.  This  includes  reliability  of  the  C2  systems  providing  the  information, 
including  withstanding  intentional  opposing  forces  disruption. 

Training 

The  effective  teaching  of  skills,  and  development  of  military  ethos  and  common 
understanding,  underpinned  by  Doctrine,  in  order  to  execute  operational  tasks  (Officers,  Non- 
Commissioned  Officers  and  Service  personnel) 

Trust 

The  belief  that  coalition  force  elements  will  follow  through  with  their  part  of  the  operations 
plan/task  and  do  so  correctly  (e.g.  Senior  commander’s  intent  executed,  or  subordinates  trust 
in  leadership). 

Understanding/Influencing  Others 

The  ability  to  take  into  account  and  make  use  of  differences  and  similarities  in  ethos, 
mindsets  and  training  (e.g.  within  own  forces,  coalition,  opposing  forces,  and  host  population) 
to  achieve  the  objective. 

Force  element  -  Either  UK,  Coalition  or  Enemy  Forces.  May  be  a  unit  of  any  size,  ranging  from  individual  to  brigade. 

1  At  the  start  of  task  (e.g.  acclimatisation)  and  during  the  operation  (e.g.  deterioration  of  fitness  over  time). 

The  shared  belief,  values,  customs  behaviours  and  artefacts  that  members  of  a  society  use  to  cope  with  and  make  sense  of 
their  world. 

Desire  will  be  influenced  by  many  factors  e.g.  perceived  task  difficulty  and  likelihood  of  success. 
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5.4  Task  4  Using  Lessons  Learned  from  Task  3,  Repeat  the  Theme  Based  Workshop 
Using  DIAMOND13  OA  Model 

On  the  basis  of  Task  3  outcomes,  the  repeat  DIAMOND  workshop  format  was  amended.  The  main 
changes  comprised: 

•  Extending  the  duration  of  the  workshop  from  one  to  two  days, 

•  Providing  a  briefing  of  the  given  OA  model  on  the  day  of  the  workshop, 

•  Clarifying  aims  and  roles  of  the  participants, 

•  Presenting  the  themes  as  a  tool  rather  than  discussion  point. 

Of  key  importance  to  the  process  was  the  identification  of  participants  with  the  relevant  HS  expertise  and 
inclusion  of  a  MA  with  recent  operational  experience.  Pivotal  to  this  was  a  sound  understanding  of  the 
model  by  the  HS  point  of  contact  through  briefing  and  opportunity  for  question  and  answer. 

Several  lessons  were  again  identified  which  informed  the  process  and  GPG.  These  included  (for  the  HS 
participants)  a  need  for  a  clarification  of  the  question  at  hand  (the  model  vs.  the  problem  space)  and 
practical  examples  of  the  model  outputs.  The  OA  participants  required  guidance  on  the  incorporation  of 
important  HF  as  assumptions.  From  a  process  perspective,  the  need  for  participants,  (including  OA 
representatives)  to  attend  all  elements  of  the  two-day  workshop,  the  importance  of  using  a  skilled 
facilitator,  and  the  need  for  the  provenance  of  the  Themes  to  be  expressed  were  all  found  to  be  key.  The 
process,  themes  and  GPG  were  amended  to  reflect  lessons  learned. 

For  the  DIAMOND  model  the  HS  participants  identified  seven  HF  areas  that  should  be  considered  for 
inclusion  in  the  model: 

•  Cultural  understanding 

•  Risk  of  cultural  bias  in  our  understanding  of  non  Blue  elements 

•  Unexpected  emergent  behaviours  as  parties  interact 

•  Ability  to  understand/discriminate  complex  information 

•  Fevel  of  trust  between  units/factions 

•  Number  of  dimensions  of  cultural  group  differences 

•  Command  Personality  or  Mission  Command. 

For  all  these  HF  areas,  the  discussion  that  they  generated  identified  whether  the  HS  community  could 
supply  data  readily  or  not,  whether  existing  information  could  be  better  exploited  to  meet  the  need  and 
whether  there  was  a  requirement  for  additional  research.  As  an  existing  model,  the  practical  issues  of 
future  funding  will  determine  how  much  additional  effort  will  be  expended  on  these  options. 

5.5  TASK  5  Test  Theme  Based  Workshop  Using  a  Warfighting  OA  Model  (COMAND14) 

Key  lessons  which  emerged  in  this  workshop  were: 

•  The  importance  of  having  an  OA  model  presentation  briefing  reviewed  by  a  HS  representative,  to 
ensure  clarity  and  understanding  by  the  HS  community 

•  The  need  to  reiterate  that  the  role  of  the  Themes  were  as  a  checklist  rather  than  a  rigid  framework 


DIAMOND  -  Diplomatic  And  Military  Operations  in  a  Non-warfighting  Domain. 
COMAND  -  C3  Orientated  Model  of  the  Air  and  Naval  Domain. 


13-10 


RTO-MP-HFM-202 


Improving  the  Representation  of  Human  Factors  in  Operational  Analysis 


•  The  need  to  allow  participants  to  discuss  priority  HF  for  inclusion  in  the  model  after  initial  voting 
to  foster  consensus. 

For  the  COMAND  model,  the  HS  participants  identified  five  HF  areas  that  should  be  considered  for 
inclusion  in  the  model: 

•  Acute  fatigue,  as  part  of  Force  Fitness 

•  Distribution  of  experience  within  the  unit,  in  terms  of  training 

•  Timeliness  of  information,  in  terms  of  availability 

•  Individuals’  understanding  of  their  own  level  of  understanding 

•  Trust  in  information. 

Again,  availability  of  funding  to  the  model  custodians  for  the  inclusion  of  these  HF  identified  is  a  key 
consideration  in  them  being  taken  forward.  However,  in  both  cases  it  was  illustrated  that  the  processes  can 
be  used  to  ensure  OA  models  are  improved,  better  linkages  made  and  better  tasking  and  exploitation  of  HF 
research  that  may  be  incorporated  into  models  in  due  course. 


6  OUTCOMES  FROM  PROCESS  PERSPECTIVE 

From  a  process  perspective  there  were  two  key  outcomes  from  the  above  workshop: 

•  The  first  is  a  table  of  themes  (shown  previously  in  Table  1)  which  acts  as  a  checklist  so  enabling 
the  HS  community  to  ensure  all  aspects  the  problem  space  are  considered  when  identifying  the 
relevant  HF 

•  The  second  is  a  revised  draft  GPG  [Ref  5;  Annex  B]  reflecting  all  the  lessons  learned  form  the 
trials  and  validation  activities.  A  key  element  of  the  draft  GPG  is  the  series  of  steps  to  deliver  a 
structured  workshop  to  identify  the  HF  in  a  given  customer  question/OA  model. 

The  facilitated  workshop  format  has  been  proven  to  be  most  useful  in  achieving  the  desired  outcome. 
Depending  on  complexity,  this  may  (and  most  often  will)  require  two  days;  it  requires  careful  facilitation 
to  ensure  engagement  by  participants,  and  to  maintain  the  pace  required  to  achieve  the  goal. 

6.1  Extract  from  the  Draft  Good  Practice  Guide 

This  extract  [Ref  4;  Annex  B]  details  the  steps  through  the  auditable  workshop  process  to  identify  relevant 
HF  in  OA  model/problem  space:  Two  facilitators  are  recommended  and  the  elements  of  the  workshop 
should  be  as  follows: 

•  OA  Briefing  This  should  include  the  fundamentals  of  OA  including  how  models  are  developed, 
how  they  are  used  (including  timeframes  etc.),  for  what  purpose  and  by  whom.  Then  an 
exploration  of  the  customer  question  should  be  addressed  including  whether  the  model  or  the 
problem  space  is  to  be  considered.  It  is  also  helpful,  where  possible  if  examples  of  the/a  model’s 
outputs  are  demonstrated.  It  is  critical  that  the  HS  POC  review  the  OA  briefing  prior  to  the 
workshop  to  ensure  Subject  Matter  Expert  understanding. 

•  Clarification  of  workshop  purpose  and  how  outcomes  will  be  used.  This  is  essential  to  ensure 
contributions  do  not  state  generalities  but  focus  on  the  actual  HF  involved.  The  understanding  that 
any  HF  proposed  will  need  to  be  supported  by  existing  theory  and/or  data,  will  focus  participants 
on  the  level  of  detail  required. 

•  Assumptions,  issues  and  definitions.  These  should  all  be  recorded  as  they  arise,  to  ensure  an  audit 
trail  of  the  process  as  well  as  allowing  discussion  and  decisions  to  be  made  at  a  later  stage. 
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•  Generation  of  HF.  The  Themes  should  be  introduced  (with  some  explanation  of  their  provenance) 
to  act  as  an  aide  memoir  and  to  ‘sort’  freely  generated  HF  relevant  to  the  question/model.  The 
themes  may  be  used  in  the  context  of  all  parties  within  the  problem  space.  This  ensures 
consideration  is  given  to  all  possible  areas  of  HF,  though  a  particular  Theme  may  be  irrelevant  to 
the  problem  space  under  scrutiny. 

•  Consolidation  HF.  The  HF  need  to  be  discussed  and  consolidated  within  each  Theme  to  identify 
the  subject  matter  of  the  topic/area.  This  can  be  done  by  the  group  directly  or  with  assistance  from 
the  facilitator,  who  can  address  any  conflicting/duplicate  topics/areas. 

•  Prioritisation  of  the  HF.  A  voting  system  to  identify  each  participant’s  top  5  or  10  key  topics/areas 
should  be  used  [depending  on  size  of  group  and  number  of  HF  generated].  Scores  should  be 
discussed  and  voted  on  iteratively  till  top  5  or  10  HF  are  identified.  Level  of  abstraction  or  impact 
on  aggregation  must  be  considered,  though  practical  issues  such  as  cost  or  data  availability  will 
also  play  a  part.  However  difficulty  of  inclusion  should  not  be  a  bar  to  the  HF  in  question;  a  record 
should  be  made  of  omissions,  and  where  possible  the  opportunity  left  open  for  its  inclusion  in  the 
model  at  a  later  date. 

•  Wav  forward.  Finally  through  discussion  on  the  availability  of  HF  data  to  inform  the 
model/question,  an  initial  outline  should  be  given  of  what  may  be  available  to  the  OA  community. 
A  discussion  can  then  be  undertaken  to  agree  what  can  be  provided  immediately,  and  what  may 
take  further  research.  Further  meetings  and  liaison  between  the  two  communities  should  be  agreed 
to  take  the  work  forward  and  ensure  coherence  and  mutual  understanding  and  manage  each 
community’s  expectations. 

It  was  intended  to  conduct  a  trial  process  to  ensure  its  universal  applicability.  However,  following  a 
review  of  the  draft  and  its  intended  audience,  the  customer  suggested  that  there  were  still  a  few  issues  with 
the  format  to  be  agreed,  and  that  the  content  should  be  applicable  to  the  wider  MOD  audience.  The 
completion  of  the  GPG  will  form  a  key  part  of  any  follow-on  task. 

The  final  sections  of  the  paper  provide  an  overview  of  the  exploitation  of  the  research  and  a  summary  of 
the  conclusions. 


7  EXPLOITATION 

Interest  in  this  study  and  its  outcomes  continues  to  be  widespread.  In  addition  to  the  presentation  at 
International  Symposium  of  Military  Operational  Research  (ISMOR)  August  2007,  international  and  pan 
UK  MOD  interest  seeks  to  apply  this  outcome  to  contexts  other  than  OA,  such  as  the  non-equipment 
Defence  Lines  of  Development  and  the  UK  MOD  Acquisition  Process. 

The  approach  has  been  employed  to  inform  a  number  of  key  UK  MOD  research  studies  including 
Operational  Degradation  model  element  of  the  Future  Capability  study.  In  addition  the  approach  has  been 
employed  in  support  to  Defence  Equipment  and  Support  (DE&S)  via  Human  Factors  Integration  Defence 
Technology  Centre  (HFI  DTC)  activities.  In  particular  this  study’s  Theme  Framework  has  been  exploited 
in  the  HFI  DTC  Social  Organisational  Framework  study,  and  the  study  team  has  provided  a  short  extract 
for  contribution  to  the  HFI  DTC  Handbook.  The  study  has  also  been  explicitly  referenced  in  support  to 
future  MOD  OA  research  studies. 


8  SUMMARY  AND  CONCLUSIONS 

This  programme  of  research  has  clearly  shown  a  journey  of  collaboration  between  two  distinct 
communities  who  talk  and  think  very  differently.  In  order  to  mitigate  the  confusion  and  frustration  that  has 


13  - 12 


RTO-MP-HFM-202 


Improving  the  Representation  of  Human  Factors  in  Operational  Analysis 


often  been  a  feature  in  past  working  relationships,  this  study  has  developed  an  approach  to  facilitate 
coherence  between  the  OA  and  HS  communities  in  their  ways  of  working  within  research.  It  has 
established  a  common  language  and  understanding  between  OA  and  HS  communities  and  together  with  an 
auditable  process  enabled  identification  and  inclusion  of  relevant  HF/Influences  in  OA.  The  approach  has 
been  refined  and  validated  using  existing  example  OA  models.  The  areas  and  principles  for  good  practice 
in  OA/HS  collaboration  have  been  identified  when  addressing  MOD  questions  and  OA  model 
development,  and  these  have  been  captured  in  a  GPG  (2nd  draft)  in  the  form  of  a  pamphlet,  intended  to  be 
trialled  with  the  Dstl  audience. 

As  stated  previously,  this  study  was  both  innovative  and  intrinsically  challenging  and  has,  by  necessity, 
had  to  change  its  direction  and  its  aims.  However,  as  a  result,  it  has  been  able  to  offer  a  more  widely 
applicable  product  (the  GPG),  whilst  still  offering  the  OA  and  HS  communities  a  framework  from  which 
to  begin  the  process  of  improving  the  representation  of  HF  in  OA  models.  In  particular'  the  research  has, 
the  established  a  process  to  bridge  the  gap  to  achieving  of  a  common  understanding  to  a  given  OA 
problem  space  between  the  OA  and  HS  communities.  It  has  also  proved  that  early  meaningful  engagement 
of  HS  in  shaping  the  OA  requirement  minimises  the  barriers  to  effective  collaboration  to  the  ultimate 
benefit  of  the  Defence  customer. 
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